Combining resampling and reweighting for faithful stochastic optimization

نویسندگان

چکیده

Many machine learning and data science tasks require solving non-convex optimization problems. When the loss function is a sum of multiple terms, popular method stochastic gradient descent. Viewed as process for sampling landscape, descent known to prefer flat minima. Though this desired certain problems such in deep learning, it causes issues when goal find global minimum, especially if minimum resides sharp valley. Illustrated with simple motivating example, we show that fundamental reason difference Lipschitz constants terms experience different variances at In order mitigate effect perform faithful optimization, propose combined resampling-reweighting scheme balance variance local minima extend general functions. We explain from numerical stability perspective how proposed more likely select true convergence analysis converges faster compared vanilla Experiments robust statistics computational chemistry are provided demonstrate theoretical findings.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Stochastic actions for diffusive dynamics: reweighting, sampling, and minimization.

In numerical studies of diffusive dynamics, two different action functionals are often used to specify the probability distribution of trajectories, one of which requires the evaluation of the second derivative of the potential in addition to the force. Here it is argued that both actions are equivalent prescriptions for the purposes of reweighting and sampling trajectories, whereas the most pr...

متن کامل

Combining RMT-based filtering with time-stamped resampling for robust portfolio optimization

Finding the optimal weights for a set of financial assets is a difficult task. The mix of real world constrains and the uncertainty derived from the fact that process is based on estimates for parameters that likely to be inaccurate, often result in poor results. This paper suggests that a combination of a filtering mechanism based on random matrix theory with time-stamped resampled evolutionar...

متن کامل

Considering Stochastic and Combinatorial Optimization

Here, issues connected with characteristic stochastic practices are considered. In the first part, the plausibility of covering the arrangements of an improvement issue on subjective subgraphs is studied. The impulse for this strategy is a state where an advancement issue must be settled as often as possible for discretionary illustrations. Then, a preprocessing stage is considered that would q...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Communications in Mathematical Sciences

سال: 2023

ISSN: ['1539-6746', '1945-0796']

DOI: https://doi.org/10.4310/cms.2023.v21.n6.a6